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Abstract 

We consider the sum of power weighted nearest neighbor distances in a sample 
of size n from a multivariate density / of possibly unbounded support. We give 
various criteria guaranteeing that this sum satisfies a law of large numbers for 
large n, correcting some inaccuracies in the literature on the way. Motivation 
comes partly from the problem of consistent estimation of certain entropies of /. 

1 Introduction 

Nearest-neighbor statistics on multidimensional data are of long-standing and contin- 
uing interest, because of their uses, for example, in density estimation and goodness-of 
fit testing [3l [HJ [21], and entropy estimation [21 HJ [9j [10] • They form a multivariate 
analog to the one-dimensional spacings statistics in which the work of S. R. Jammala- 
madaka, the dedicatee of this paper, has featured prominently. For example, [16] uses 
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nearest neighbor balls to generalize the maximum spacings method to high dimensions 
and to establish consistency in estimation questions. 

In the present note we revisit, extend and correct some of the laws of large numbers 
concerned with sums of power-weighted nearest-neighbor distances that have appeared 
in recent papers, notably Penrose and Yukich [15], Wade [19], Leonenko et al. [10J. 

Fix d G N and j G N. Given a finite X C R d , and given a point x G X, let card(A') 
denote the number of elements of X, and let D(x, X) := Dj(x, X) denote the Euclidean 
distance from x to its jth nearest neighbor in the point set X \ {x}, if card(X) > j; set 
D(x, X) := if card^) < j. Let / be a probability density funticon on M. d , and let 
(Xj)jgN be a sequence of independent random d- vectors with common density /. For 
nGN, let X n := {Xi, . . . , X n }. Let a G R and set 

n 

Sn,a := Yl X n)T = ^(^^(X, *n))°. 

x£X n i=l 

Certain transformations of the S n)Q have been proposed [HJ [10] as estimators for certain 
entropies of the density / which are defined in terms of the integrals 

W) ■= I f(y) p dy (p > o). 

For p > with p ^ 1, the Tsallis p-entropy (or Havrda and Charvat p-entropy [7]) 
of the density / is defined by H p (f) := (1 — I p (f))/(1 — p), while the Renyi entropy 
m of f is defined by H* p (f) := log/ p (/)/(l - p). 

Renyi and Tsallis entropies figure in various scientific disciplines, being used in 
dimension estimation and the study of nonlinear Fokker-Planck equations, fractal ran- 
dom walks, parameter estimation in semi-parametric modeling, and data compression 
(see [1] and [LU\ for further details and references). 

A problem of interest is to estimate the Renyi and Tsallis entropies, or equivalently, 
the integrals I p (f), given only the sample {Xj}™ =1 and their pairwise distances. Let 
Ud := 7r ci / 2 /r(l + d/2) denote the volume of the unit radius Euclidean ball in d dimen- 
sions, and set j(d, j) := UJ^ d ( ^rfi^ J- This note provides sufficient conditions on 
the density / establishing that j(d, j) -1 ^ -1 ^^ converges to Ii- a /d(f) i n L 1 , or in L 2 . 
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In other words, since L 1 convergence implies convergence of means, we provide suffi- 
cient conditions on / guaranteeing that j(d, j)~ 1 n~' 1 S nj0l is an asymptotically unbiased 
and consistent estimator of I\- a /d{f)- 

2 Results 

Two of our results can be stated without further ado. 

Theorem 2.1 Let a > 0. Suppose the support of f is a finite union of convex bounded 
sets with nonempty interior, and f is bounded away from zero and infinity on its 
support. Then as n — > oo we have L 2 and almost sure convergence 



Theorem 2.2 Let q = 1 or q = 2. Let a G (—d/q, 0) and suppose f is bounded. Then 
(\2.1\i holds with L q convergence. 

For the interesting case when a > and / has unbounded support, our results 
require further notation. Let | • | denote the Euclidean norm on W d . For r > 0, define 
the integral 



If r < s and M s (f) < oo, then M r (f) < oo. Hence M r (f) < oo for r < r c (f) and 
M r (f) = oo for r>r c (f). 

For fceN, let denote the annular shell centered around the origin of R d with 
inner radius 2 k and outer radius 2 fc+1 , and let A be the ball centered at the origin with 
radius 2. For Borel measurable A C R d , set F(A) := P[X X E A] — J A f(x)dx. 

We can now state the rest of our results. 




(2.1) 




and define the critical moment r c (f) e [0, oo], by 



r c (f) := sup{r > : M r (f) < oo}. 
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Theorem 2.3 Let q = 1 or q = 2. Let a G (0,d/q). Suppose Ji_ a /d(/) < oo, and 
r c(/) > qctd/(d — qa). Then (\2. 1\ holds with L q convergence. 

We shall deduce from Theorem 12.31 that when f(x) decays as a power of \x\, the 
condition Ii- a /d{f) < oo is sufficient for L 1 convergence: 

Corollary 2.1 Suppose there exists (3 > d such that f(x) = Q(\x\~@) as \x\ — ■> oo, i.e. 
such that for some finite positive C we have 

C~ x \x\-P < f{x) < C\x\~ fi , VxeR d , \x\>C. (2.2) 

Suppose also that Ji_ a /d(/) < oo for some a G (0, d). Then ( \2. II) holds with L l 
convergence. 

Our final result shows that in general, the condition I\- a /d{f) < oo is not sufficient 
alone for L l convergence, or even for convergence of expectations. It can also be viewed 
as a partial converse to Theorem 12.31 showing, under the additional regularity condition 
(I2.3p . that when q — 1 the condition r c (/) > qadjid — qa) is close to being sharp. 

Theorem 2.4 Let < a < d. Then (i) if r c (f) < ad/(d — a), and also for some 
k G N we have 

F(A k ) F(A k ) 
< mf — — < sup — — < oo, (2.3) 

k>k F(i4 fc _i) ~ k >k F{A k ^) 

then limsup^^ E [n~ l S n ^ = oo; 

(ii) for < r < adjid — a) there exists a bounded continuous density func- 
tion f on ~R d satisfying (\2.3h . such that Ii- a /d(f) < oo? but with r c (f) = r so that 
limsup^^ E [n~ 1 S n , a \ = oo by part (i). 



The value of the limit in (12.11) was already known (see Lemma I3~T|) . The contribution 
of the present paper is concerned with the conditions under which the convergence (12.11) 
holds; in what follows we compare our conditions with the existing ones in the literature 



4 



and also comment on related limit results. For conditions under which n~ 1 ' 2 (S n;a — 
ES n J is asymptotically Gaussian, we refer to [HI [T| fT2]. 
Remarks. 

(i) Theorem \2.1\ The condition in Theorem 12.11 is a slight relaxation of condition 
CI of the L 2 convergence results in [15] or [19], which assume a polyhedral support 
set. When the support of / is the unit cube, Theorem 2.2 of [8] gives an alternative 
proof of almost sure convergence in ( 12. lj) (we remark that Theorem 2.2 of [8] contains 
an extraneous E in the left-hand side) . The convergence of means implied by Theorem 
12.11 was previously obtained, under some extra differentiability conditions on /, in [5]. 

(ii) Theorem \2.2\ The L 1 convergence of Theorem 12.21 improves upon Theorem 
3.1 of [10], which establishes mean convergence; the L 2 convergence of Theorem 12.21 is 
contained in Theorem 3.2 of [10] and we include this for completeness. 

(iii) Theorem \2.3[ The condition in Theorem 12.31 corrects the condition of the 
corresponding result given [15J, where for L l convergence it is stated that we need 
r c (f) > d/(d— a); in fact we need instead the condition r c (f) > ad/(d — a). In the 
proof of Theorem 12.31 below, we shall indicate the errors in the proof in [15] giving rise 
to this discrepancy. This correction also applies to condition C2 in Theorem 2 of [19], 
the proof of which relies on the result stated in |15j . 

(iv) Theorem \2.J\ The condition ( 12.31) holds, for example, if f(x) is a regularly 
varying function of \x\. Given ( I2.3P and given I\- a /d < °°i Theorem 12.41 shows that 
the condition r c (/) > adjid — a) is necessary for L 1 convergence of n~ 1 S Uta , while 
Theorem 12.31 says that r c (f) > ad/(d — a) is sufficient. It would be of interest to try 
to find more refined necessary and sufficient conditions when r c (f) = ad/(d — a). 

(v) General <p. For : R+ -> R+ put S n ^ := J2 xe x n ( t ) ( nl/dD ( x ^n))- If <P has 
polynomial growth of order a, that is if there is a constant a G (0, oo) such that 
<f)(x) < C(l + x a ) for all x G M + , then straightforward modifications of the proofs show 
that under the conditions of Theorem 12.11 or Theorem 12.31 we have the corresponding 
L q convergence 

n-'Sn,^ [ E[(f>(D{0,V m ))]f(x)dx, 
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where for all r > 0, V T is a homogeneous Poisson point process in M. d having constant 
intensity r, and D(0,V T ) is the distance between the origin of M. d and its jth nearest 
neighbor in V T . 

(vi) Minimal spanning trees. Given a finite X cM. d and <p : M + — > M + , let 



where MST(A') denotes the edges in the graph of the minimal spanning tree on X. 
Thus L,p(X) is the sum of the 0- weighted edge lengths in the minimal spanning tree 
on X. Let q = 1 or 2. If has polynomial growth of order a, with a G (0,d/q), if 
h-a/d(f) < oo, and if r c (f) > qad/(d — qa) then, as may be seen by following the 
proof of Theorem l2.3l the proof of Theorem 2.3(iii) of [15] in fact shows that as n —>■ oo 
we have 



where the convergence is in L q , and where MST(0,Vf( x )) denotes the edges in the 



this is new whereas for q = 1 and a G (0, 1), this improves upon Theorem 2.3(iii) of 
[T5] . which requires r c (f) > max(a<i/ (d — a), d/ (d — a)). 

(vii) Non-existence of density. If the {Xi\™ =l fail to have a density, then normal- 
ization of S n ^ a may involve exotic functions of n, including log periodic normalizations, 
as is the case when the {Xi}f =1 have a Cantor distribution on [0, 1]; see [T8]. 

(viii) Comparison with fT^. The convergence of expectations corresponding to (12.11) 
is given as the main conclusion in Theorem 3.1 of [10]. In the case 1 — a/d < 1 of 
that result, it is claimed that this convergence of expectations holds without any extra 
conditions besides finiteness of h- a /d- Theorem 12.41 here disproves this assertion; the 
argument in [10] requires that convergence in distribution implies convergence of rth 
moments, which is not in general true. On the other hand, Corollary 12.11 shows that 
if we assume f(x) decays as some power of |x| then finiteness of I\- a /d is indeed a 
sufficient condition for convergence in L l , and hence also convergence of expectations. 



L+(X) := Yl 



eeMST(X) 




minimal spanning tree graph on U Vf( x ) incident to 0, the origin of M. d . When q = 2, 
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3 Proofs 



This section provides the proofs of the results stated in the preceding section. We 
denote by c, C, C, and C" various strictly positive finite constants whose values may 
change from line to line. The proofs of Theorems I2.1[ 12.21 and 12.31 use the following 
result. 

Lemma 3.1 Letq G {1,2} and a G R. Suppose for some p > q that E [(n 1 ' d D(Xi, A' n )) ap ] 
is a bounded function ofn. Then (\2. l\i holds with L q convergence. 

Proof. Since D is a stabilizing functional on homogeneous Poisson point processes 
|15j . we can apply Theorem 2.2 of [15J or Theorem 2.1 of [15] to get L q convergence 
of n~ l S nta to a limit which is expressed as an integrated expectation in [15J (see eqn 
(2.15) of P3]). It was shown in [19] that this limit is equal to the right hand side of 
(12. ip (and this is also consistent with the limiting constant in [5]). □ 

Proof of Theorem \2.1\ Recall that we assume the support of /, namely supp(/) : = 
{x G M. d : f(x) > 0}, is a finite union of bounded convex sets with nonempty interior, 
here denoted B 1 ,...,B m . Set A := sup{|x — y\ : x G supp(/),?/ G supp(/)}, the 
diameter of the support of /. By assumption, A < oo. Also we assert that there is a 
constant c > such that for r G (0, A], 

F(B r (x)) > cr d , Vx G supp(/), (3.1) 

To see this, take S\ > such that for 1 < i < m there is a ball of radius 5i contained 
in Bi. There is a constant 82 > such that for 1 < i < m, if x G B i} and r < 5i, then 
the intersection of the ball of radius r centered at x with the convex hull of the union 
of B^ and x has volume at least 52r d . This region is contained in Bi and (13.11) follows 
for r G (0, d>i]. But then (with a different choice of c) (13. ip follows for r < A. Hence, 
for < t < \n l l d and with B(x; r) denoting the Euclidean ball of radius r centered at 
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X, 



P[n l/d D(X l , X n ) >t}< sup P[card(A' n _ 1 n B(x; n - 1/d t)) < j] 

xEsupp(/) 

i=0 ^ ^ ' 

< C ^ f d exp(-cn- V(n - 1 - z)) 
< C"(l +t (j - 1)d )exp(-ct d ) < C"exp(-(c/2)t d ). 

Moreover this probability is clearly zero for t > \n l l d . Hence, for a > and p > 2, 

/■oo 

E [(n 1/d D(Xi, A'„)) Qp ] = / P[n 1/d D(X u X n ) > u 1/{ap) ]du 

Jo 

POD 

<C exp(-(c/2)u d/ ( ap) )du 
Jo 

which is finite and does not depend on n. Therefore we can apply Lemma 13.11 to get 
the L 2 convergence (12.11) . 

For almost sure convergence, we apply Theorem 2.2 of [13] . where here the test 
function considered in that result (and denoted / there, not to be confused with the 
notation / as used here) is the identity function. It is well known (see [3], or Lemma 
8.4 of [20]) that there is a constant C := C(d) such that for any finite X C M. d , any 
point x G X is the jth nearest neighbor of at most C other points of X . Therefore 
adding one point to a set X within the bounded region supp(/) changes the sum of 
the power- weighted jth nearest neighbor distances by at most a constant. Therefore 
(2.9) of p2] holds here (with (3 = 1 and p' = 5 say), and the almost sure convergence 
follows by Theorem 2.2 of p2]. □ 

Proof of Theorem \2.2l The proof depends on the following lemma. Recall that 
(Xj)j>i are i.i.d. with density /. Given X\, let V n denote the volume of the d- 
dimensional ball centered at n 1 ^ d X 1 whose radius equals the distance to the jth nearest 
point in n l l d (X n \ X\) } where for r > and X C M d we write rX for {rx : x G X}. For 
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all x G M. d , for all n = 2, 3, ... and for all v G (0, oo) let 

F niX (v):=P\y n <v\X 1 = x]. (3.2) 
Lemma 3.2 // / zs bounded and S G (0, 1), then 



f oo 

sup E V~ s = sup / / v~ 5 dF njX (v)f(x)dx < oo. 

n n jRd Jo 



Proof of Lemma \3. e A Since /g 00 ^ p dF{y) = p / °°f p 1 F{y)dv for any p G (0,1) 
whenever both integrals exist (see e.g. Lemma 1 on p. 150 of |6j), we have for all 



oo poo 

Sjtp /„. \ r / „.-5-l 



x g 

/■ex 

/ v- a dF n>x (v) = 5 I v- d - l F n;X (v)dv 
Jo Jo 

pi poo pi 

< / v- & - 1 F U;X (v)dv + 5 \ v - & - l dv= \ v- s - l F n:X (v)dv + l. 
Jo Ji Jo 

With B v (x) denoting the ball of volume v around x, for all v G (0, 1) we have 

F n , x (v) = P[V n < v\X x = x] = l- P[card(r2 1 /^ n _ 1 n B v {n l,d x)) < j] 

< 1 - P[card(n 1/d A' n _ 1 n B v (n 1/d x)) = 0] 

(\ n-1 
1- / f(z)dz) . (3.3) 
JB v/n (x) J 

Since / is assumed bounded we have 

F n>x (v) < 1 - exp ((n - 1) log(l - \\f\\oov/n)) . 

When n is large enough, for all v G (0, 1) we have (n— 1) log(l — 1|/|| oof/ft.) > — 211/Hooi;, 
and so for all x G M. d 

F n , x ( v ) < 1 - exp(-2||/|| 00 t;) < 2||/ 

Hence for all n large enough and all x we have v~ s ~ x F niX {v)dv < 2||/||oo f v~ s ~ x vdv : 
demonstrating Lemma [3. 2[ □ 



Now to prove Theorem 12. 2\ we choose p > q such that — 1 < ap/d <0 and invoke 
Lemma 13.21 to conclude sup n E [Vn p/d ] < oo. We now apply Lemma 13.11 to complete 
the proof of L q convergence. □ 

The proof of Theorem 12.31 uses the following lemma. Recall from Section [2] the 
definition of the regions A k , k > 0. 

Lemma 3.3 Let < s < d. Ifr c {f) > sd/(d - s), then J2T=i 2 ks (F(A k )Y d ~ s ^l d < oo. 

Proof. We modify some of the arguments on page 85 of [20]. For all e > 0, by 
Holder's inequality we have 

^22 ks (F[A k ]) id - s)/d = ^2- £ks (F[A k }) { - d - s),d 2 { - l+£)ks 

k k 

(\ s/d / \ (d-s)/d 

J2(2- £ks ) d / s ) ( F[A k ]{2 {l+£)ks ) d/{d ' s) 

(d~s)/d 



<C(e } s) Ij2 J \x\ {1+£)sd/{d - s) f(x)dx 



k JA k 

which, for e small enough, is finite by hypothesis. □ 

Proof of Theorem \2.3[ We follow the proof in [15], but correct it in some places and 
give more details in others. We aim to use Lemma [3.11 Since we assume < a < d/q, 
we can take p > q with ap < d. Clearly 



E {(n lld D{X x , X n )) ap ] = n^^E 



n ap/d-l E [L^iXn)], (3.4) 



.1=1 

where for any finite point set X C M rf , and any b > 0, we write L b (X) for ~^2 xeX D(x, X) b 
(and set L b (0) := 0). Note that for some finite constant C = C(d,j) the functional 
X i— > L b (X) satisfies the simple subadditivity relation 

L b {x uy) < L b (x) + L b (y) + ct b (3.5) 

for all t > and all finite X and y contained in [0 } t] d (cf. (2.2) of [5D]). 
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As in (7.21) of [20J or (2.21) of p2] we have that 

L ap {X n ) < n A k ) ) + C(p) max \X t 



op 



(3.6) 



vfc = 



In the last sentence of the proof of Theorem 2.4 of [15J it is asserted that the last term 
in ( 13. 6j) is not needed, based on a further assertion that one can take C = in ( 13.51) 
here, but these assertions are incorrect. For example, if card(^) < j then L b (y) = 
but l}{X U y) could be strictly greater than L b (X). Similarly, if card(A" n fl A k ) < j 
then the term in (13.61) from that k is zero but the corresponding contribution to the 
left side of ( 13.61) is non-zero. 

Combining (I3.6P with (13.41) yields 



E[(n 1/d D(X 1 ,X n )) ap ) <n {ap - d)/d E 



Y,L ap (x n nA k 



+C(p)E[n {ap - d)/d max|Xi 



(3.7) 



By Jensen's inequality and the growth bounds L ap (X) < C(diamAf) Qp (card(A'))^ a P)/ d 
(see Lemma 3.3 of [20]), we can bound the first term in the right hand side of (13.71) by 



C J22 kap (F[A k }) 



(d—cep)/d 



(3i 



Recall that we are assuming < a < d/q and also r c (f) > qda/(d — qa) (the last 
assumption did not feature in [15], but in fact we do need it). Let p > q be chosen so 
that r c (f) > dapj{d — ap) as well as ap < d. Setting s = ap in Lemma l3TB"| we get 
that the expression ( 13.81) is finite. Thus the first term in the right hand side of ( 13. 71) is 
bounded by a constant independent of n. 

The second term in the right hand side of (13.71) is bounded by 



C(p) 



P 



max \XA ap > tn {d ~ ap)/d 

Ki<n 



•oo 

dt+ I P 



max \XA ap > tn {d - ap)/d 

Ki<n 



dt 



< C{p) [l+n p[\X 1 \ * d K dr - c *) > t d/{d - ap) n]dt 



By Markov's inequality together with the assumption r c (f) > dap/(d — ap), this last 
integral is bounded by a constant independent of n. 
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Therefore the expression (13.71) is bounded independently of n, so we can apply 
Lemma [3. II to get the L q convergence in (12.11) . □ 



Proof of Corollary \2.1\ Suppose for some (3 > d that f(x) = 0(|a;| _/3 ) as \x\ — > oo. 
Then it is easily verified that given a G (0, d), the condition Ii- a /d(f) < 00 implies 
that —/3(l — at/d) + d < and hence (3 > d 2 (d — a)~ 1 . Moreover, it is also easily checked 
that r c (f) = (3 — d so that if (3 > d 2 (d — a)" 1 then r c (f) > dajid — a). 

Therefore, if I\- a /d{f) < oo we can apply the case q = 1 of Theorem 12.31 to get 
(12. ip with L 1 convergence. □ 

The proof of Theorem 12.31 shows that 

E[(n 1/d D(X 1 ,X n )) £ ] < oo, (3.9) 

if e > is such that ed/(d — e) < r c (f). The proof of Theorem 12.41 given below, shows 
that the condition ed/(d — e) < r c (f) cannot be dropped in general. 



Proof of Theorem 2-4 Let < a < d. Suppose that r c (f) < ad/ (d — a), and ( 12.31) 



holds for some ko G N. Choose r,s such that r c (f) < r < s < adjid — a). Then 
M r {f) = oo, so ^ fc 2 rfc F(A fc ) = oo and therefore there is an infinite subsequence /C of 
PJ such that 

2 sk F{A k ) > 1, kelC. (3.10) 

Indeed, if no such fC existed, then for all but finitely many k we would have 2 rk F(A k ) < 
2( r - s ) k which is summable in k. 

Given k G N, and set n(k) = [(F(ylfc)) _1 ] , the smallest integer not less than 
(F(A k )y 1 . Let E k be the event that X x G A k but X, L £ A k _ x U A k U A k+l for 2 < i < 
n(k). Then by the condition (12.31) . there is a strictly positive constant c, independent 
of k, such that for k > ko we have 

P[E k ) = F{A k ){\ - F(A k ^ UA k U Ak+x))^' 1 > cF(A k ). 
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If Ek occurs then D(X h X n{k) ) > 2 k ~\ so for n = n(k) we have (for a different constant 
c) that 

E [n-'S^] = E [{n 1/d D{X x , X n )) a ] > n a ' d E [D(X U X n ) a l Ek ] 

> cn a/d F{A k )2 ka > c{F{A k )y- a/d 2 ka . 

By (I3.10p . for k E K, this lower bound is at least a constant times 2 k ( a ~ s ( d ~ a ^ d \ and 
therefore tends to infinity as k — > oo through the sequence fC, concluding the proof of 
part (i). 

For part (ii), for each k > 2 choose, in an arbitrary way, a unit radius ball B k that 
is contained in A k . Given r G (0,ad/(d — a)), consider the density function / with 
f(x) = C2~ rk for x G B k ,k > 2, and with f(x) = for x G M d \ ^T =2 B k, here the 
normalizing constant C is chosen to make / a probability density function. This gives 
F(A k ) = Cucfi,~ rk for each k > 2; it is easy to see that this / has r c (f) = r, and that 
( 12. 3p holds with k = 3. Also, for any p > we have I p (f) = u d C p J2k>2 2~ rpk which is 
finite, so in particular Ii- a /d < oo- This choice of / is bounded but not continuous, but 
can easily be modified to a continuous density with the same properties, for example 
by modifying / in an annulus near the boundary of each ball B k so as to make it 
continuous, and then adjusting the normalizing constant C accordingly. □ 
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